Two field incidents show an _nstat_rwlock/UDP socket mutex ABBA deadlock followed by a configd watchdog panic. Raw wait-owner records and UUID-matched KDK stacks identify the same cycle in each incident's earlier configd diagnostic and later panic.
Affected systems
Both kernels: xnu-13432.1.9~1. Kernel UUIDs are in the attachments.
Report AReport BmacOS27.0 (26A428)27.0.1 (26A434)HardwareMac16,10 / T8132Mac15,6 / T6030CloudShell PID87443906Blocked configd threadPID 341, TID 19794593PID 113, TID 309370Our NEDNSProxyProvider can capture CloudShell's DNS UDP traffic and forward it via a separate upstream socket. Provider flows and CloudShell sockets have separate lifetimes.
Symptoms and reproduction
During DNS UDP forwarding and flow/socket cleanup, CloudShell blocks in final socket close. configd's IPConfigurationAgentQueue blocks creating a UDP socket: a diagnostic is captured after 60 seconds without check-ins, followed by panic/restart after 180 seconds:
userspace watchdog timeout: no successful checkins from configd (1 induced crashes) in 180 secondsExpected: socket close completes or returns an error. We have no deterministic reproducer, measured failure rate, or reconstructed per-flow close history.
Direct lock-owner evidence
Raw waitInfo records (mutex tokens are opaque):
Report A, 26A428:
thread 18691491: kernel mutex 0x61c831d7f38edbc3 owned by thread 19491850
thread 19491850: krwlock 0xfffffe000cb41730 for writing owned by thread 18691491
thread 19794593: krwlock 0xfffffe000cb41730 for writing owned by thread 18691491Report B, 26A434:
thread 36984: kernel mutex 0x44aa649ef3095ae1 owned by thread 12555
thread 12555: krwlock 0xfffffe000ca01730 for writing owned by thread 36984
thread 309370: krwlock 0xfffffe000ca01730 for writing owned by thread 36984In each incident, the first two records form a cycle in both snapshots; the third is configd. All three threads are TH_WAIT / TH_UNINT.
UUID-matched kernel stacks
Caller-to-callee chains; listed offsets match in both kernels.
Kernel worker (A: TID 18691491; B: TID 36984):
nstat_idle_check + 808
-> in_pcb_checkstate + 128
-> udp_lock + 200
-> lck_mtx_lock_contended + 1212Holds _nstat_rwlock exclusively; waits for CloudShell's UDP socket mutex.
CloudShell (A: TID 19491850; B: TID 12555):
soclose + 212
-> soclose_locked + 2300
-> sofreelastref + 60
-> flow_divert_detach + 332
-> in_pcbdetach + 168
-> nstat_pcb_detach + 92
-> lck_rw_lock_exclusive + 180
-> lck_rw_lock_exclusive_gen + 912Holds the socket mutex; waits for _nstat_rwlock. XNU acquires these locks, not application code.
configd (downstream victim):
socket_common -> socreate_internal -> udp_attach
-> nstat_udp_new_pcb -> lck_rw_lock_exclusivePossible enabling condition
Matched 27 KDK disassembly shows a provider CLOSE path through cfil_sock_is_dead that directly marks the PCB DEAD and schedules GC without nstat cleanup. Public xnu-12377.121.6 (26.5) uses ordinary detach first. Provider CLOSE and final fd close are distinct.
A DEAD PCB can retain an nstat locus/reference. Under _nstat_rwlock, nstat_idle_check removes the locus and releases the reference via in_pcb_checkstate(..., WNT_RELEASE, FALSE), which can take the socket mutex. Final close takes the reverse order; nstat_pcb_detach requests the nstat lock even when the locus is null.
This is a possible enabling path from disassembly/source analysis, not proof of the preceding provider CLOSE in either incident. The older source already has opposite static lock orders; 26.5 is not proven immune.
Questions for Apple
Our provider calls closeReadWithError:nil and closeWriteWithError:nil when finished, as documented for NEAppProxyFlow. This does not validate the entire asynchronous lifecycle.
Is this cycle known, and is there a fixed build?
How is reverse locking avoided after the direct DEAD transition?
Is there a supported workaround preserving flow cleanup? Close rate-limiting is not proven to eliminate the issue.
Which diagnostics would help correlate provider CLOSE, DEAD, and final fd close?
Attachments and timestamps
Four EXCERPT attachments contain watchdog messages, raw frames, KDK annotations, and wait-owner records. Full reports are retained for private Feedback Assistant submission.
26A428_panic_excerpt.txt
26A428_configd_excerpt.txt
26A434_panic_excerpt.txt
26A434_configd_excerpt.txt
2026-10-15, +0800: configd capture -> embedded panic event:
A: 04:24:43.1258 -> 04:26:44
B: 08:49:29.4349 -> 08:51:32
Use embedded panic times, not file/header timestamps. Original pairs match on boot identity; identifiers are omitted from excerpts.
Sources: NEAppProxyFlow, XNU 26.5 comparison.